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Abstract 



A population of firing neurons is expected to carry information not only 
by mean firing rate but also by fluctuation and synchrony among neurons. 
In order to examine this possibility, we have studied responses of neuronal 
ensembles to three kinds of inputs: mean-, fluctuation- and synchrony-driven 
inputs. The generalized rate-code model including additive and multiplica- 
tive noise (H. Hasegawa, Phys. Rev. E 75 (2007) 051904) has been studied 
by direct simulations (DSs) and the augmented moment method (AMM) in 
which equations of motion for mean firing rate, fluctuation and synchrony are 
, derived. Results calculated by the AMM are in good agreement with those 

qt 1 by DSs. The independent component analysis (ICA) of our results has shown 

that mean firing rate, fluctuation (or variability) and synchrony may carry in- 
dependent information in the population rate-code model. The input-output 
relation of mean firing rates is shown to have higher sensitivity for larger mul- 
tiplicative noise, as recently observed in prefrontal cortex. A comparison is 
ff") . made between results obtained by the integrate-and-fire (IF) model and our 

rate-code model. 
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1 Introduction 



One of the most important and difficult problems in neuroscience is to understand 
how neurons communicate in a brain. There has been a long-standing controversy 
between the temporal- and rate-code hypotheses in which information is assumed to 
be encoded in firing timings and rates, respectively [1]- [3]. A recent success in brain- 
machine interface (BMI) [HE], however, suggests that the population rate code is 
employed in sensory and motor neurons, though it is still controversial which of rate, 
temporal or other codes is adopted in higher-level cortical neurons. 

In recent years, much attention has been paid to a study on effects of mean firing 
rate, fluctuation and synchrony (or spatial correlation) of input signals (for a review 
on rate and synchrony, see Ref. [6]). The precise role of synchrony in information 
transmission and the relation among the firing rate, fluctuation and synchrony are 
not clear at the moment [6]- [12]. The firing rate and synchrony are reported to 
be simultaneously modulated by different signals. For example, in motor tasks of 
monkey, firing rate and synchrony are considered to encode behavioral events and 
cognitive events, respectively [7]. During visual tasks, rate and synchrony are sug- 
gested to encode task- related signals and expectation, respectively [8]. A change in 
synchrony may amplify behaviorally relevant signals in V4 of monkey [9]. An in- 
crease in synchrony of input signals is expected to yield an increase in output firing 
rate. The synchrony of neurons in extrastriate visual cortex is, however, reported to 
be modulated by selective attention even when there is only small change in firing 
rate [12]. Rate-independent modulations in synchrony are linked to expectation, 
attention and livalry [6] . Fluctuations of input signals have been reported to modify 
the / — / relation between an applied dc current / and autonomous firing frequency 
/ although its sensitivity to input fluctuation seems to depend on a kind of neurons 
[13j-[15j. The / — I curve of prefrontal cortex (PFC) retains the increased sensi- 
tivity to input fluctuations at large J, while that of somatosensory cortex (SSC) is 
insensitive to input fluctuation though its linearity is increased at small / [15]. 

This kind of problems discussed above have been extensively studied by using 
spiking neuron models such as the Hodgkin-Huxley model JTH] and integrate-and-fire 
(IF) model with diffusion and mean-field approximations [UJ-[33]. The purpose of 
the present paper is to examine the same problem by using the rate-code model, 
which is an alternative theoretical model to the spiking model. In a previous paper 
[31] (referred to as I hereafter), we proposed the generalized rate-code model for 
coupled neuron ensembles with finite populations, which are subjected to additive 
and multiplicative noises. It seems natural to include multiplicative noise beside 
additive noise in our rate-code model because the noise intensity is expected to 
generally depend on the state of neurons. Actually effects of multiplicative noise 
in the spiking neuron model are extensively examined by using conductance-based 
inputs which yield multiplicative noise [32]" [HZ]- Our calculation in I has shown that 
the introduced multiplicative noise leads to the non-Gaussian stationary distribu- 
tion of firing rate, yielding interspike-interval (ISI) distributions such as the gamma, 
inverse-Gaussian and log-normal distributions, which have been experimentally ob- 
served. We discussed in I the dynamical properties of neuron population, by using 
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the augmented moment method (AMM) which was developed for a study of stochas- 
tic systems with finite populations [38]. In the AMM, we pay attention to global 
properties of neuronal ensembles, taking account of mean and fluctuations of lo- 
cal and global variables. The AMM has the same purpose to effectively study the 
properties of neuronal ensembles as approaches based on the population-code hy- 
pothesis [3H]-[I3]- The AMM has been nicely applied to various subjects of neuronal 
ensembles [331 145] and complex networks [46J . 

We have assumed in I that input signals are the same for all neurons in the 
ensemble. In the present study, input signals are allowed to fluctuate and to be 
spatially correlated. We will derive equations of motion for mean, fluctuation and 
synchrony of firing rates with the use of the AMM, in order to investigate the 
response to mean-, fluctuation- and synchrony- driven inputs. This study clarifies, 
to some extent, their respective roles in information transmission. 

The paper is organized as follows. In Sec. 2, we discuss an application of the 
AMM to the generalized rate model, studying the input-output relations of mean, 
fluctuation and synchrony. In Sec. 3, stationary and dynamical properties are 
discussed with numerical model calculations. By using the independent component 
analysis (ICA) [47J, we investigate a separation of signals when two or three kinds 
of inputs are simultaneously applied. Discussions are presented in Sec. 4, where 
the results obtained by spiking IF model are compared with those by our rate-code 
model. The final Sec. 5 is devoted to conclusion. 



2 Formulation 

2.1 Adopted model 

For a study of the properties of a neuron ensemble containing finite N neurons, we 
have adopted the generalized rate-code model [341 145] in which a neuron is regarded 
as a transducer from input to output signals, both of which are expressed in terms 
of firing rates. The dynamics of the firing rate r^(t) (> 0) of a neuron i (i — 1 to 
N) is given by 



dt 

with 



dv ■ 

= F{r i ) + H{u l ) + G{ ri ) m {t)+ &(t), (1) 



«*(*) = (Vl E rAV+m, (2) 

1 z J m) 



ii 

H(u) = -_=e(n). (3) 
V w + 1 

Here F(r) and G(r) are arbitrary functions of r, Z (= N — 1) denotes the coordina- 
tion number, 7j(t) an input signal from external sources, w the coupling strength, 
and Q(u) the Heaviside function: 0(it) = 1 for u > and Q(u) — otherwise. Addi- 
tive and multiplicative noises are included by £j(t) and rji{t), respectively, expressing 
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zero-mean Gaussian white noise with correlations given by 



(Vi(t) Vj(t')) = a 2 fe + ci(l-y^-f'), (4) 
(&(*)£#)) = (3 2 [5 lJ +c (l-5 l3 )}5(t-t f ), (5) 
(m(t)Ut')) = 0, (6) 

where the bracket (•) denotes the average over the distribution of p({ri},t) [Eq. 
([All) ], a {(3) stands for the magnitudes of multiplicative (additive) noise, and c\ 
(co) expresses the degree of the spatial correlation in multiplicative (additive) noise. 
The gain function H{u) in Eq. expresses the response of a rate output (r) to 
a rate input (u). It has been shown that, when spike inputs with mean firing rate 
Ti are applied to a Hodgkin-Huxley neuron, mean firing rate r a of output signals is 
r a ~ Ti for Ti ~ 60 [Hz], above which r a shows the saturation behavior HS] • The 
nonlinear, saturating behavior in H(u) arises from the fact that a neuron cannot 
fire with the rate of r > l/r r (= r max ) where r r denotes the refractory period. The 
function H(u) has the rectifying property because the firing rate is positive, which is 
expressed by the Heaviside function in Eq. ([3]). Although our results to be present 
in the following are valid for any choice of H(x), we have adopted, in this study, a 
simple analytic expression given by Eq. ([3]), where the rate is normalized by r max . 

With the use of the diffusion-type approximation, a spatially correlated input 
signal Iiit) in Eq. (T5]) is assumed to be given by 

hit) = mW + siitt), (7) 

with 

(6Ii(t)) = 0, (8) 

{SlitySItf)) = 7/(t)[<J - + (l-<J -)W*-*') 5 ( 9 ) 
where variance (7/) and covariance (5/7/) are given by 

nit) = ^£(^(t) 2 >, (10) 

i 

5/(*)7i(*) = ^EE(»j(i)), (11) 

Si(t) expressing the degree of the spatial correlation. We will discuss responses of 
the neuronal ensemble described by Eqs. ([I])-© to the spatially correlated input 
given by Eqs. dZD-flSJ) with given ///, 7/ and Sj, by using both direct simulations 
(DSs) and the AMM [32 [38] • 

2.2 Augmented moment method 

In the AMM |38j, we define the three quantities of //, 7 and p given by 

Mt) = W)> = -^ !>*(*)>. ( 12 ) 
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7(*) = ^EW*)-M*)] a >, (13) 

= 4eE(W^)-M*)][^)-^)]>> (1 4 ) 

« 3 

where R = (l/N)J2i r i, (i>{t) expresses the mean, and j(t) and p(t) denote the 
averaged, auto and mutual correlations, respectively, of firing rates. 

By using the Fokker-Planck equation (FPE), we obtain equations of motion for 
(rj) and (r^j) (for details see appendix A). Expanding in Eqs. ( 1A2[) and (IA3I) 
around the average value of /i as 

r i (t)=^(*)+<Jr i (*)» ( 15 ) 

and retaining up to the order of (SriSrj), we obtain equations of motion for p, 7 and 
p. AMM equations in the Stratonovich representation are given by (the argument t 
being suppressed) 

dp f cx 2 \ 

= fo + h + f 2 7+ f — J [0o0i + 3(0i</ 2 + 0003)7], ( 16 ) 

% = 2f ll + ^(Np- 1 ) + 2(g 2 + 2g g 2 )a 2 1 + ll + a 2 g 2 + {3 2 , (17) 



dp 



= 2f lP + 2h lW p + 2(g{ + 2g g 2 )a 2 p 

+ —(ji + a 2 gl + (3 2 ) + ^{liSi + Cl a 2 gl + c /3 2 ), 



where £ = (l/£!)(d t F(n)/dx t ), gt = (l/£\)(d e G(p)/dx e ), h e = (l/£\)(d e H(u)/du £ ) 
and u = wp + pi. Original A-dimensional stochastic DEs given by Eqs. CO)-® are 
transformed to the three-dimensional deterministic DEs given by Eqs. (jT6|) - (fl8l) . 
For 7 j = Si = cq = c\ = 0, equations of motion given by Eqs. (fIB]) - (fl8|) reduce to 
those obtained in our previous study [34] . From p, 7 and p obtained by Eqs. (fT6|) - 
( [181) . we may calculate important quantities of synchrony and variability. Then, 
they may be expressed in physically more transparent forms, as will be discussed in 
the following [Eqs. (j23} - (j25jl ]. 



2.3 Rate synchronization and variability 
2.3.1 Synchronization ratio 

The synchronization is conventionally discussed for firing timings (temporal synchro- 
nization) or phase (phase synchronization) . We discuss, in this paper, the synchro- 
nization for firing rate (rate synchronization) . In order to quantitatively discuss the 
synchronization, we first consider the quantity P(t) given by 

P(t) = < Ut) - rM 2 >= 2[ 7 («) -p{t)]- (19) 

ij 
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When all neurons are firing with the same rate (the completely synchronous state), 
we obtain rj(t) = R(t) for all i, and then P(t) = in Eq. (jT9l) . On the contrary, 
we obtain P(t) = 2(1 — l/N)~/(t) = Po(t) in the asynchronous state where p = 7/iV 
[34, 38J. We may define the normalized ratio for the synchrony of firing rates given 
by PI 

S(t) is and 1 for completely asynchronous (P = Po) and synchronous states 
(P = 0), respectively. 

2.3.2 Variability 

The variability in the ISI is usually defined by by Cy = \fy^ / p T where p T and 7 T 
stand for mean and variance of ISI, respectively. We here define the rate variability 
given by 



= jm = m (21) 

IX fi{t) 



Similarly, the variability in input rate signals is given by 



pi Hiit) 

We may show that if the temporal firing rate is given by rj(t) = 1/Tj(£), we obtain 
Cv(t) — Cy(t), where Tj(i) denotes the ISI of firing times in a neuron i. 

2.4 AMM equations for fi, 7 and S 

It is noted that the variability and synchrony are related to the second-order statis- 
tics of local (7) and global fluctuations (p), respectively, of firing rates. Employing 
the relations given by Eq. (1201) . we may transform equations of motion for /1, 7 and 
p given by Eqs. (TTdT) - (TT5|) to those for /1, 7 and S given by 

-g = fo + h + hi + ( — j [</offi + %i#2 + ^0^3)7] , (23) 
^ = 2f ll + 2h 1 w 1 S + 2(g 2 1 +2g g 2 )a 2 1 + ll + a 2 g 2 +(3 2 } (24) 
f = —(li + a'g' + ^S+^y^Sj + ^gl + coP 2 ) 

+ (^f)(l + ZS)(l-S). (25) 

Equations (!23l) -(j25l) express a response of (p, 7, 5) to a given input of (///, 77, S/). In- 
put signals in which information is encoded in pi, 7/ and S\ are hereafter referred to 
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as mean-driven, fluctuation-driven and synchrony-driven inputs, respectively. Equa- 
tions (123]) - (125]) show that 7/ plays the same role as independent noise while 7/5"/ as 
correlated noise. 

When we adopt F(r) and G(r) given by 



F(r) = -\r a , (26) 
G(r) = r\ (a,b>0) (27) 



Eqs. (j23D-d25D become 



^ = -A/T + h - (£j a{a - lK- 2 7 



+ [j) [b^ 1 + Kb - 1)(26 - 1)^7], (28) 

^ = -2Aa y u a - 1 7 + 2/i 1 w 7 5 + 26(26- l)aV fe " 2 7 + 7/ + «V b + /5 2 , (29) 

^ = --(7/ + «V b + /? 2 )5+f-V7/^ + Ci«V 6 + c /5 2 ) 
dt 7 V7/ 

+ (^yi + ZS)(l-S), (30) 

where A expresses the relaxation rate. We note that for (i) a = or 1 and (ii) 6 = 0, 
1/2 or 1, a motion of \x is decoupled from the rest of variables because the a(a — 1) 
and b(b — l)(2b — 1) in the third and fourth terms of Eq. ( 128]) vanish. Equation ( 130] ) 
shows that a motion of S is ostensibly independent of the index a of F(r) although 
it depends on a through 7. 



3 Model Calculations 

3.1 Stationary properties 

In order to get an insight to the present method, we will show some model calcula- 
tions. When we consider a special case of a = b = 1.0 in Eqs. (1261) and 



F(r) = -Ar, (31) 
G(r) = r, (32) 

Eqs. fl28l - fl30l are expressed by 

dp a 2 fi 

— = -\n + h + —-, 33 
at 2 

= -2A7 + 2h x wyS + 2a 2 7 + 7/ + a V + (3 2 , (34) 

~tt = --(7/ + aV + /5 2 ) + -(7/^/ + ciaV + c /3 2 ) 
at 7 7 

+ + (35) 
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The stationary solution of Eqs. (!33|) - (l35|) is given by 

" = (A-aV2)' (36) 

(7/ + aV + ^) ( ~ 7) 

7 2(A-a 2 -/i lW 5)' 1 J 
9 Z(A - a 2 ) (7/5/ + cia 2 // 2 + c /3 2 ) + ^(7/ + a 2 p 2 + /3 2 ) 

( 7/ + a 2 /i 2 + /? 2 )[Z(A-« 2 )-/ il w(Z-l)] + /i 1 w;7 / ^ ' [ } 



(jiSi + c x a 2 p 2 + cp/J 2 ) 
(7/ + «V + /5 2 ) ' 

[Z(\- a 2 ) -h lW (Z 



for w = (39) 
for c = ci = S/ = (40) 



We note in Eq. ( 1361) that /i is increased as pi is increased with an enhancement 
factor of 1/(A — a 2 /2), but // is independent of 77 and S7- Local fluctuation 7 is 
increased with increasing input fluctuation (7/) and/or noise (a, (3) as Eq. (1371) 
shows. Equation (I38I) shows that 5 is increased with increasing Sj, c , Ci and/or w, 
as expected. 

The stability condition around the stationary state may be examined from eigen- 
values of the Jacobian matrix of Eqs. fl33l) - fl35l) . which are given by (for details, see 
appendix B) 



a 2 



Ai = -X + — + h lW} (41) 



2hiiu 



A 2 = -2A + 2a 2 (42) 

A 3 = -2A + 2cv 2 + 2/i!W. (43) 

The first eigenvalue of Ai arises from an equation of motion for /z, which is decoupled 
from the rest of variables. The stability condition for \x is given by 

hiw < (A - a 2 /2). (44) 

In contrast, the stability condition for 7 and p is given by 

-Z(A-« 2 ) < h x w < (A -a 2 ). (45) 

Then for A — a 2 < h\w < A — « 2 /2, 7 and p are unstable but p remains stable. 

The parameters in our model are a, (3, w and iV: we hereafter set A = 1.0 
and Ci = c = 0.0 to reduce the number of model parameters. Input signals are 
characterized by pi, 77 and Si. We will present some numerical calculations for 
mean-, fluctuation- and synchrony-driven inputs which are calculated with the use 
of Eqs. (J36D-([38D for a = 0.0, (3 = 0.1 and N = 100. 

A. Mean-driven inputs 

Figure [1] shows the pi dependences of p and S for w = 0.0 (dashed curves) and 
w = 0.5 (solid curves) with 7/ = 0.2 and Sj = 0.2. We note that for w — 0, p is 
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increased with increasing \xj after the gain function of H(u), while S is independent 
of hi. In contrast, for finite w, fi is much increased than that for w = 0. The chain 
line expressing \i = /if crosses the \i curve at \i = for w = 0.0 and at \i — 0.735 
for w = 0.5, 

B. Fluctuation-driven inputs 

Figure [2] shows Cy and S against Cyi (= y/ji/^i) for w = 0.0 (dashed curves) 
and w = 0.5 (solid curves) with \ii = 0.2 and Si = 0.2. For w = 0.0, an increase in 
Cyi yields an increase in Cy and S, while \i is independent of CVi, as Eqs. (I36l) - (l38l) 
show. The chain curve shows Cy = Cyf. the region where Cy < Cyi is realized for 
Cyi > 0.51 for w = 0.0 and C VI > 0.22 for w = 0.5. 

C. Synchrony-driven inputs 

Figure [3] shows the Si dependences of \i and S for w = 0.0 (dashed curves) and 
w = 0.5 (solid curves) with \ii = 0.2 and 7/ = 0.2. With increasing Si, S is increased 
as expected. For w = 0, \i is independent of Sj, as Eqs. shows. For iu = 0.5, S 
is much increased compared to that for w = 0.0. 

It is necessary to point out that the /xj dependence of \x is modified by multi- 
plicative noise (a). An example of the /ij dependence of /1 is plotted in Fig. 0]for 
various a. With increasing a, \x shows a steeper increase for larger a because of the 
(A — « 2 /2) factor in Eq. ( |36l) . This reminds us the recent experiment of prefrontal 
cortex (PFC) showing that the / — I curve has the increased sensitivity at large 
/ with increasing input fluctuation [15]. This is interpreted as due to a shorten, 
effective refractory period by fluctuation in the calculation using the IF model [15] . 

The dependence of S on Si is plotted in Fig. [5] for various values of iV (a = 0.5, 
(3 = 0.2, w = 0.5). It is shown that the synchrony S is more increased in smaller 
system. The result for iV = 100 is nearly the same as that for iV = 00. 

3.2 Dynamical properties 
3.2.1 Pulse inputs 

In order to study the dynamical properties of the neuronal ensemble given by Eqs. 

we have performed DSs by using the Heun method [50l [51] with a time 
step of 0.0001: DS results are averages of 100 trials. AMM calculations have been 
performed for Eqs. (!3"3l - (l3~5l) by using the fourth-order Runge-Kutta method with 
a time step of 0.01. We consider, as a typical example, an ensemble with A = 1.0, 
a = 0.1, j3 = 0.1, w = 0.5 and N = 100. If we adopt a larger N in our model 
calculations, fluctuations in DS results are decreased and we may obtain smoother 
results, although it needs much computational times. Among /1, 7 and S, the 
strongest iV dependence is realized in S [Eqs. fl36|) - fl38|) ]. Figure [5] shows that the 
synchrony S almost saturates at iV > 100 where it has little N dependence. This 
is a reason why we have adopted iV = 100 for our model calculations. Calculated 
responses to mean-, fluctuation- and synchrony- driven pulse inputs are shown in 
Figs. El H and [HJ respectively, where solid and dashed curves show results of the 
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AMM and DS, respectively. 



A. Mean-driven inputs 

First we apply a mean-driven pulse input given by 

Hj(t) = A Q(t - 40)0(60 — t) + A b , (46) 

with A = 0.4, A b = 0.1, 7/(t) = 0.1 and Sj(t) = 0.1. Responses of fi(t), j(t), S(t) 
and Cv(t) calculated by the AMM (solid curves) and DS (dashed curves) are shown 
in Figs. Eta), (b), (c) and (d), respectively: input signals of ///(£), 7i(i) and Sj{t) are 
plotted by chain curves in (a), (b) and (c), respectively. An increase in an applied 
mean-driven input at 40 < t < 60 induces an increase in n(t) and decreases in ~f(t) 
and S(t) which arise from the h\ term in Eqs. ( 1351) . By an applied pulse input, 
Cy(t) is decreased because of the increased \x. The results of the AMM are in fairly 
good agreement with those of DS. 

B Fluctuation-driven inputs 

Next we apply a fluctuation-driven input given by 

7j(*) =BQ(t- 40)0(60 - t) + B b , (47) 

with B = 0.2, B b = 0.05, /i^t) = 0.1 and Si(t) = 0.1. Figures [3(a)- (d) show 
calculated responses of fi(t), j(t), S(t) and Cy(t). When the magnitude of 7/(t) 
is increased at 40 < t < 60, 7(t) and Cy(t) are much increased, while there is no 
changes in //(£). S(t) is modified only at t ~ 40 and t ~ 60, where the input pulse 
is on and off. 

c. Synchrony-driven inputs 

We apply a synchrony-driven input given by 

S r (t) =Ce(t- 40)0(60 -t) + C b , (48) 

with C = 0.4, C b = 0.1, Hi(t) = 0.1 and ji(t) = 0.1, whose results are plotted in Fig. 
E(a)-(d). An increase in synchrony- driven input at 40 < t < 60 induces increases in 
S(t), 7(t) and Cy(t), but no changes in fi{t). This is because fi(t) is decoupled from 
the rest of variables in Eqs. fl33l) - fl35l) for the case of F(r) = —Xr and G{r) = r. 

3.3 Independent component analysis 

It is interesting to estimate multivariate input signals from multiple output signals. 
Such a procedure has been provided in various methods such as Bayesian estimation 
and ICA [47]. Here we consider ICA, which was originally developed for a linear 
mixing system, and then it has been extended to linear and nonlinear dynamical 
systems. ICA has revealed many interesting applications in various fields such as 
biological signals and image processing. A vector x of output signals is a real function 
F of a vector s of input sources: 

x=F(s). (49) 
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The dimension of s is assumed to be the same or smaller than that of x. If components 
of s are statistically independent and if only one of the source signals is allowed to 
have a Gaussian distribution, ICA may extract a vector y with a function G given 
by 

y = G(x), (50) 
from which we may estimate the original source as s ~ y |47J. 

A. Coexistence of fij and Sj 

We will discuss the case when mean- and synchrony- driven inputs are simultane- 
ously applied to the neuronal model. We consider the mean-driven sinusoidal input 
and synchrony-driven toothsaw input, given by 

///(t) = 0.1 [1 - cos(2tt£/20)] + 0.1, (51) 
Sj(t) = 0.01mod(t,50), (52) 

with 7/(t) = 0.1 where mod(a, b) denotes the mod function expressing the residue 
of a divided by b. Two panels in Fig. [5^a) show input signals of /ir(t) and Si(t), 
and two panels in Fig. Mb) show output signals of /x(£) and S(t) calculated by the 
AMM. We note a little distortion in S(t) due to a cross talk from fi(t). Assuming 
s = (fMj, SiY and x = (/x, S)\ we have made an analysis of our result by using 
ICA. Two panels in Fig. M^c) show two components of y extracted from output 
signals of x = (//, Sy shown in Fig. [9](b) with the use of the fast ICA program [52J. 
Although the program is designed for linear, mixing signals, we have employed it for 
our qualitative discussion. We note that results in Fig. [H^c) fairly well reproduce 
the original, sinusoidal and toothsaw signals shown in Fig. E]Ja). 

B. Coexistence of /xj, 77- and Si 

Next we consider the case where three kinds of inputs are simultaneously ap- 
plied. They are mean-driven sinusoidal signal, fluctuation-driven toothsaw signal 



and synchrony-driven square pulse signal, given by 

///(t) = 0.1 [1 - cos(27rf/20)] + 0.1, (53) 

7/(t) = 0.002 mod(t, 50), (54) 

S T (t) = 0.5 6[-cos(2vrt/120)]. (55) 



Three panels in Fig. HPT a) show the input signals of /xj(t), Ji(t) and Sj(i). Output 
signals of /x(i), 7(t) and S(t) calculated in the AMM are shown in three panels of 
Fig. ITOTb). where j(t) and S(t) are a little distorted by a cross talk. We have made 
an ICA analysis of our result, assuming s = (/x/,7/, S/)^ and x = (/i, 7, S)t. Three 
panels of Fig. [TOTc) show signals extracted by ICA. Extracted sinusoidal and square 
signals in Fig. [TUlfc) are similar to those of input signals shown in Fig. [TOT a), though 
the fidelity of a toothsaw signal is not satisfactory. This is partly due to the fact 
that the fast ICA program adopted in our analysis is developed for linear mixing 
models, but not for dynamical nonlinear models [52] . 

These ICA analyses show that the mean, fluctuation and synchrony may inde- 
pendently carry information in our population rate-code model. 
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4 Discussion 



4.1 A comparison with previous studies 

Various attempts have been proposed to obtain the firing-rate model, starting from 
spiking neuron models [53j-[57J. It is difficult to analytically calculate the firing rate 
based on the firing model, except for the IF-type model [19]-[33j. Calculations with 
the use of the IF model have shown the followings: 

(1) increased input firing rate deceases output variability [32 J, 

(2) increased input firing rate decreases synchrony [20l [22] , 

(3) increased input fluctuation raises firing rate [T5l [26]. 

(4) increased input synchrony increases firing rates [Bl E21 ESI E21 [21] , and 

(5) increased input synchrony increases variability |21j . 

The items (1), (2) and (5) are consistent with our result shown in Figs. [6] and [71 In 
contrast, items (3) and (4) seem to inconsistent with our result showing that fi(t) 
is independent of 7r(t) and Si(t), as given by Eqs. (J33l . It is noted, however, that 
the model calculation given in the preceding section has been made for the case of 
F(r) = —Xr and G{r) = r, in which a motion of n(t) is decoupled from those of 
7(t) and S(t). This is not the case in general. We note in Eq. f[28l that fi{t) is not 
decoupled from j(t) and S(t) except for the case in which (a = or 1) and {b = 0, 
1/2 or 1). For example, in the case of F(r) = —Xr and G(r) = r 2 , equations of 
motion given by Eqs. (J28|) -(l30l) become 



~dt 
~dt 



— A/i + ho + a 2 (/i 3 + 3/ry), (56) 
-2A7 + 2h 1 wyS + 12a 2 /i 2 7 + 7/ + "V + P 2 , (57) 



^ = --(7/ + «V + /5 2 )5+-( 7 ^ + S i«V + co/3 2 ) 
at 7 7 

+ (^f){l + ZS){l-S\ (58) 

Equations (!56l) - (!58|) clearly show that is coupled with j(t) and S(t). 

Figure [TT1 shows time courses of fJ,(t), j(t) and S(t) calculated with the use of 
Eqs. (|56l) - (l58l) when a mean-driven input fii(t) given by Eq. ( 1461) with A = 0.4, 
At, = 0.1, 7/(t) = 0.2 and S/(t) = 0.2 is applied to a neuron ensemble with a = 0.35, 
P = 0.1,w = 0.5, d = c = 0.0 and N = 100. We note that 7(f) is modified by 
an applied at 40 < t < 60, showing the coupling between /x(£) and j(t) in the 
case of F(r) = —Xr and G(r) = r 2 , while there is no couplings between them in the 
case of F(r) = —Xr and G(r) = r as shown in Fig. 

A number of neuronal experiments have not reported systematic a change in 
firing rates while the synchronization within an area is modulated [6] . In particular, 
the synchrony is modified without a change in firing rate in some experiments 
[9j [11]. It has been pointed out that such phenomenon may be accounted for by a 
mechanism of a rapid activation of a few selected interneurons [12]. Recently the 
absence of a change in firing rate is shown to be explained if the ratio of excitatory 
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to inhibitory synaptic weights of long-range couplings is kept constant in neuron 
ensembles described by the IF model [33J. 



5 Conclusion 

We have discussed the rate code of firings neuron population, studying the re- 
sponses of the generalized rate-code model [3UH5] to three kinds of inputs: mean-, 
fluctuation- and synchrony-driven inputs. The ICA analyses of our results have sug- 
gested that mean rate, fluctuation (or variability) and synchrony may carry inde- 
pendent information. It would be interesting to examine this possibility by neuronal 
experiments using in vivo or in vitro neuron ensembles. 

One of advantages of our rate-code model given by Eqs. ([I])-© is that we can 
easily discuss various properties of neuronal ensembles, by changing F(r), G(r), 
H(u) and model parameters. It is possible to theoretically examine various cases 
of F(r) and G(r) in a systematic way. We hope that our rate-code model shares 
advantages with phenomenological neuronal models such as the Hopfield [58] and 
Wilson-Cowan models [59J. 

The Tsallis and Fisher information entropies are very important quantities ex- 
pressing information measures in nonextensive systems [00] ■ It is challenging to 
calculate information entropies in neuronal ensembles with spatially correlated vari- 
ability [01] ■ The plasticity of synapses (depression and facilitation) is known to play 
important roles in the activity of neurons. Indeed, a memory in brain is accounted 
for by the plasticity of synapses in the Hopfied model [58J. A variety of activity 
in prefrontal cortex in response to sensory stimuli is expected to be explained by 
dynamic synapses. It is promising to take into account dynamic synapses in our 
approach, which is left as our future study. 
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Appendix A: Derivation of AMM equations given by Eqs. 

(HSD-dlSD 

The Fokker-Planck equation (FPE) for the Langevin equation given by Eq. ([T]) 
in the Stratonovich representation is given by 



j t p({r k }, f) = - E j-{[F{ ri ) + H( Ui )} p({r k }, t)} 

4EE^{feH(i-yKHt)}, 

% j t 3 
ry^ f) f) 

+y E Efci + c i(! - M gr^M^-IGfa) P ({r k }, «)]}. (Al) 

i j 1 j 
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Equations of motion for moments, (r^) and (r^ rj), are derived with the use of FPE 
( ' {r ' } { F {r i ) + H{u i )) + ^{G'{r l )G{r i )), (A2) 



,11 ty/ 2 

= (r, \F{r 3 ) + fffa)]) + [F(rO + #(«,)]) 



+ y[(r J G'(r,)G(r,)) + (r J G'(r l )G(r J ))] 
+ ^.[ 7/ + a 2 (G(r l ) 2 )+/5 2 ] 

+ (1 - <%) [7/Si + Cl a 2 (G(r,)G(r,)) + c (3 2 }. (A3) 
We may obtain Eqs. f|T6l) - f|T8l) . by using the expansion given by 

n = n + Sn, (A4) 



and the relations given by 



dp 1 ^ d{ri) 

Tt ~ iV^^T' (A5j 
dl 1 ^ d((5ri) 2 } 

Tt ~ iv^^^' (A6) 

dp 1 dJJhjSr^) 

Tt ~ Jp\\^t~- { } 

For example, Eq. (fl6l) for dfi/dt is obtained as follows. 

^E(^)) = /0 + /27, (A8) 
iE(%)> = ^0, (A9) 

i 

^E( G 'W G (^)> = ^1 + 3(^3 + ^2)7- (Aio) 

i 

Equations (fT7j) and ( Tl8l) are obtainable in a similar way By using Eq. (J20l) . we 
obtain equations of motion for pit), 7 (£) and S(£) given by Eqs. (I23l -( l25l) . 

Appendix B: Jacobian matrix of Eqs. (1331)- (|35l) 

In making a linear stability analysis, it is better to adopt the basis of (p, 7, p) 
than that of (p, 7, S). In the former basis, equations of motion for F(r) = — Xr and 
G{r) = r given by Eqs. (fT6l)-(fT8l) become 

i = -\? + ho + -f, (Bl) 
I = _ 2A7 + ^( p _^) +2a27 + 7/ + aV + ^, (B2) 
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= -2Xp + 2h 1 wp + 2a 2 p+^(7/ + «V + (3 2 ) 

+ jjiriiSi + Cl «V + c /3 2 ), (B3) 

The stability of the stationary state may be examined by the Jacobian matrix of 
Eqs. (B1)-(B3). With the use of C12 = C13 = C32 = in the matrix, we obtain its 
eigenvalues given by 

a 2 

Ai = cn = -A + — + h lW , (B4) 
A 2 = C22 = _2A + 2a 2 -^, (B5) 

Zj 

A3 = c 33 = -2A + 2a 2 + 2h lW , (B6) 
from which the stability condition given by Eqs. ( 1441) and ( 1451) is obtained. 
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Figure 1: (Color online) Stationary values of /i and S as a function of \ij for w = 0.0 
(dashed curves) and w = 0.5 (solid curves) with 7/ = 0.2 and Si = 0.1 (A = 1.0, 
a = 0.0, (3 = 0.1, and N = 100), the chain curve expressing \i = ///. 



Figure 2: (Color online) Stationary values of Cy and S as a function of Cy/ for 
w = 0.0 (dashed curves) and w = 0.5 (solid curves) with /// = 0.2 and SV = 0.2 
(A = 1.0, a = 0.0, (3 = 0.1, and iV = 100), the chain curve expressing Cy = Cyj. 



Figure 3: (Color online) Stationary values of /1 and S as a function of Si for w = 0.0 
(dashed curves) and w = 0.5 (solid curves) with fii = 0.2 and 7/ = 0.2 (A = 1.0, 
a = 0.1, (3 = 0.0, and N = 100), the chain curve expressing S = Sj. 



Figure 4: fi as a function of for various a values with A = 1.0 and w = 0.0. 



Figure 5: S as a function of Si for various iV with /!/ = 0.1, 7/ = 0.0, A = 1.0, 
a = 0.5, /3 = 0.2 and w = 0.5. 



Figure 6: (Color online) Time courses of (a) fi(t), (b) j(t), (c) S(t) and (d) Cy(t) 
for mean-driven pulse input Hi(t) given by Eq. (j46l (A = 0.4, A;, = 0.1) with 
7/(t) =0.1 and Si(t) =0.1, calculated by the AMM (solid curves) and DS (dashed 
curves): chain curves in (a), (b) and (c) express inputs of /i/(t), Ji(t) and Si(t), 
respectively (A = 1.0, a = 0.1, = 0.1, w = 0.5 and N = 100). 



Figure 7: (Color online) Time courses of (a) /i(t), (b) "f(t), (c) S(t) and (d) Cy(t) 
for fluctuation-driven pulse input 7/(t) given by Eq. (T4T1) (5 = 0.2, £? b = 0.05) with 
Hi{t) = 0.1 and Si(t) = 0.1, calculated by the AMM (solid curves) and DS (dashed 
curves): chain curves in (a), (b) and (c) express inputs of fii(t), 7/(t) and Si(t), 
respectively (A = 1.0, a = 0.1, (3 = 0.1, w = 0.5 and N = 100). 
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Figure 8: (Color online) Time courses of (a) //(£), (b) j(t), (c) S(t) and (d) Cy(t) 
for synchrony-driven pulse input Si(t) given by Eq. (I4"8"j) (C = 0.4, Cf, = 0.1) with 
Hi(t) = 0.1 and 7/(t) = 0.1 calculated by the AMM (solid curves) and DS (dashed 
curves): chain curves in (a), (b) and (c) express inputs of /i/(t), 7/(i) and Sj(t), 
respectively (A = 1.0, a = 0.1, (3 = 0.1, w = 0.5 and N = 100). 



Figure 9: ICA separation of the AMM result for input signals where mean- and 
synchrony-driven inputs coexist: (a) input signals, (b) output signals, and (c) ex- 
tracted signals by fast ICA [52J (A = 1.0, a = 0.1, (3 = 0.1, w = 0.5 and N = 100). 



Figure 10: ICA separation of the AMM result for input signals where mean-, 
fluctuation- and synchrony- driven inputs coexist: (a) input signals, (b) output sig- 
nals, and (c) extracted signals by fast ICA [52] (A = 1.0, a = 0.1, (3 = 0.1, w = 0.5 
and N = 100). 



Figure 11: (Color online) Time courses of (a) fi(t), (b) j(t), (c) S(t) and (d) Cyit) 
for mean-driven pulse input fi(t) given by Eq. ( 1461) (A = 0.4, Ab = 0.1) with 
7j(t) = 0.2 and Sj(t) = 0.2 calculated by the AMM (solid curves) for F(r) = -Xr 
and G{r) = r 2 with Eqs. fl56l)- fl58|) : chain curves in (a), (b) and (c) express inputs 
of /ii(t), 7/(t) and Si(t), respectively (A = 1.0, a = 0.4, (3 = 0.1, w = 0.5 and 
N = 100). 
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